<!DOCTYPE html>


<html lang="zh-CN">


<head>
  <meta charset="utf-8" />
    
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
  <title>
    怎么理解linux的平均负载及平均负载高后的排查工具.md |  
  </title>
  <meta name="generator" content="hexo-theme-ayer">
  
  <link rel="shortcut icon" href="/favicon.ico" />
  
  
<link rel="stylesheet" href="/dist/main.css">

  
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/Shen-Yu/cdn/css/remixicon.min.css">

  
<link rel="stylesheet" href="/css/custom.css">

  
  
<script src="https://cdn.jsdelivr.net/npm/pace-js@1.0.2/pace.min.js"></script>

  
  

  

</head>

</html>

<body>
  <div id="app">
    
      
    <main class="content on">
      <section class="outer">
  <article
  id="post-linux/怎么理解linux的平均负载及平均负载高后的排查工具"
  class="article article-type-post"
  itemscope
  itemprop="blogPost"
  data-scroll-reveal
>
  <div class="article-inner">
    
    <header class="article-header">
       
<h1 class="article-title sea-center" style="border-left:0" itemprop="name">
  怎么理解linux的平均负载及平均负载高后的排查工具.md
</h1>
 

    </header>
     
    <div class="article-meta">
      <a href="/2020/11/11/linux/%E6%80%8E%E4%B9%88%E7%90%86%E8%A7%A3linux%E7%9A%84%E5%B9%B3%E5%9D%87%E8%B4%9F%E8%BD%BD%E5%8F%8A%E5%B9%B3%E5%9D%87%E8%B4%9F%E8%BD%BD%E9%AB%98%E5%90%8E%E7%9A%84%E6%8E%92%E6%9F%A5%E5%B7%A5%E5%85%B7/" class="article-date">
  <time datetime="2020-11-10T16:00:00.000Z" itemprop="datePublished">2020-11-11</time>
</a> 
  <div class="article-category">
    <a class="article-category-link" href="/categories/linux/">linux</a>
  </div>
  
<div class="word_count">
    <span class="post-time">
        <span class="post-meta-item-icon">
            <i class="ri-quill-pen-line"></i>
            <span class="post-meta-item-text"> 字数统计:</span>
            <span class="post-count">2.4k</span>
        </span>
    </span>

    <span class="post-time">
        &nbsp; | &nbsp;
        <span class="post-meta-item-icon">
            <i class="ri-book-open-line"></i>
            <span class="post-meta-item-text"> 阅读时长≈</span>
            <span class="post-count">11 分钟</span>
        </span>
    </span>
</div>
 
    </div>
      
    <div class="tocbot"></div>




  
    <div class="article-entry" itemprop="articleBody">
       
  <h1 id="怎么理解linux的平均负载及平均负载高后的排查工具"><a href="#怎么理解linux的平均负载及平均负载高后的排查工具" class="headerlink" title="怎么理解linux的平均负载及平均负载高后的排查工具"></a>怎么理解linux的平均负载及平均负载高后的排查工具</h1><h2 id="什么是平均负载"><a href="#什么是平均负载" class="headerlink" title="什么是平均负载"></a>什么是平均负载</h2><p>平均负载可以对于我们来说及熟悉又陌生，但我们问平均负载是什么，但大部分人都回答说平均负载不就是单位时间内CPU使用率吗？其实并不是这样的，如果可以的话，可以 man uptime 来了解一下平均负载的详细信息。</p>
<p>简单的说平均负载是指单位时间内，系统处于<strong>可运行状态和不可中断状态</strong>的平均进程数，也就是说<strong>平均活跃进程数</strong>，它和CPU使用率并没有直接关系。这里解释一下可运行状态和不可中断这两个词。</p>
<p>可运行状态：</p>
<ul>
<li>指正在使用CPU或者正在等待CPU的进程，我们使用ps命令查看处于R状态的进程</li>
</ul>
<p>不可中断状态：</p>
<ul>
<li>进程则是正处于内核态关键流程中的进程，并且这些流程是不可中断的。例如：常见的等待硬件设备I/O的响应，也就是我们在ps命令查看处于D状态的进程</li>
</ul>
<p>比如，当一个进程向磁盘读写数据时，为了保证数据的一致性，在得到磁盘回复前，它是不能被其他进程中断或者打断的，这个时候的进程处于不可中断状态，如果此时的进程被打断了，就容易出现磁盘数据和进程数据不一致的问题。</p>
<p>所以，<strong>不可中断状态实际上是系统进程和硬件设备的一种保护机制。</strong></p>
<p>因此，你可以简单理解为，平均负载就是平均活跃进程数。平均活跃进程数，直观上的理解就是单位时间内的活跃进程数，但它实际上是活跃进程数的指数衰减平均值。既然是平均活跃进程数，那么理想状态，就是每个CPU上都刚好运行着一个进程，这样每个CPU都会得到充分的利用。例如平均负载为2时，意味着什么呢？</p>
<ul>
<li>在只有2个CPU的系统上，意味着所有的CPU刚好被完全占用</li>
<li>在4个CPU的系统上，意味着CPU有50%的空闲</li>
<li>而在只有1个CPU的系统上，则意味着有一半的进程竞争不到CPU</li>
</ul>
<h2 id="平均负载和CPU使用率"><a href="#平均负载和CPU使用率" class="headerlink" title="平均负载和CPU使用率"></a>平均负载和CPU使用率</h2><p>现实工作中，我们经常容易把平均负载和CPU使用率混淆，所以在这里，我也做一个分区。</p>
<p>可能你会疑惑，既然平均负载代表的是活跃进程数，那平均负载高了，不就意味着CPU使用率高吗？</p>
<p>我们还是要回到平均负载的含义上来，平均负载是指单位时间内，处于可运行状态和不可中断状态的进程数，所以，它不仅包括了正常使用CPU的进程，还包括了等待CPU和等待I/O的进程。</p>
<p>而CPU使用率，是单位时间内CPU的繁忙情况的统计，跟平均负载并不一定完全对应，例如：</p>
<ul>
<li>CPU密集型进程，使用大量CPU会导致平均负载升高，此时这两者是一致的</li>
<li>I/O密集型进程，等待I/O也会导致平均负载升高，但CPU使用率不一定很高</li>
<li>大量等待CPU的进程调度也会导致平均负载升高，此时的CPU使用率会很高</li>
</ul>
<h2 id="平均负载案例"><a href="#平均负载案例" class="headerlink" title="平均负载案例"></a>平均负载案例</h2><p>这里我们需要安装几个工具sysstat、stress、stress-ng</p>
<p>这里Centos的sysstat版本会老一点，最好升级到最新版本。手动rpm安装或者源码安装</p>
<h3 id="场景一、CPU密集型"><a href="#场景一、CPU密集型" class="headerlink" title="场景一、CPU密集型"></a>场景一、CPU密集型</h3><p>1、运行一个stress命令，模拟一个CPU使用率100%场景</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ stress --cpu 1 --timeout 600</span><br></pre></td></tr></table></figure>

<p>2、开启第二个终端，uptime查看平均负载的变化情况</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ watch -d uptime</span><br><span class="line"> 09:40:35 up 80 days, 18:41,  2 users,  load average: 1.62, 1.10, 0.87</span><br></pre></td></tr></table></figure>

<p>3、开启第三个终端，mpstat 查看CPU使用率的变化情况</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ mpstat -P ALL 5 2010:06:37 AM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle10:06:42 AM  all   31.50    0.00    0.35    0.00    0.00    0.00    0.00    0.00    0.00   68.1510:06:42 AM    0    1.20    0.00    0.80    0.00    0.00    0.00    0.00    0.00    0.00   98.0010:06:42 AM    1    7.21    0.00    0.40    0.00    0.00    0.00    0.00    0.00    0.00   92.3810:06:42 AM    2  100.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.0010:06:42 AM    3   17.43    0.00    0.20    0.00    0.00    0.00    0.00    0.00    0.00   82.36</span><br><span class="line"># -P ALL 表示监控所有CPU，后面数字5 表示间隔5秒输出一次数据</span><br></pre></td></tr></table></figure>

<p>从第二个终端可以看到，1分钟平均负载增加到1.62，从第三个终端我们可以看到有一个CPU使用率100%，但iowait为0，这说明平均负载的升高正式由CPU使用率为100%</p>
<p>那我们查看是那个进程导致了CPU使用率为100%呢？我们可以使用pidstat来查看：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">#每5秒输出一次数据$ pidstat -u 5 1 </span><br><span class="line">10:08:41 AM   UID       PID    %usr %system  %guest   %wait    %CPU   CPU  Command</span><br><span class="line">10:08:46 AM     0         1    0.20    0.00    0.00    0.00    0.20     0  systemd</span><br><span class="line">10:08:46 AM     0       599    0.00    1.00    0.00    0.20    1.00     0  systemd-journal</span><br><span class="line">10:08:46 AM     0      1043    0.60    0.00    0.00    0.00    0.60     0  rsyslogd</span><br><span class="line">10:08:46 AM     0      6863  100.00    0.00    0.00    0.00  100.00     3  stress</span><br><span class="line">10:08:46 AM     0      7303    0.20    0.20    0.00    0.00    0.40     2  pidstat</span><br></pre></td></tr></table></figure>

<p>从这里我们可以看到是stress这个进程导致的。</p>
<h3 id="场景二、I-O密集型进程"><a href="#场景二、I-O密集型进程" class="headerlink" title="场景二、I/O密集型进程"></a>场景二、I/O密集型进程</h3><p>1、我们使用stress-ng命令，但这次模拟I/O压力，既不停执行sync:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">#--hdd表示读写临时文件#-i 生成几个worker循环调用sync()产生io压力</span><br><span class="line">$ stress-ng -i 4 --hdd 1 --timeout 600</span><br></pre></td></tr></table></figure>

<p>2、开启第二个终端运行uptime查看平均负载情况</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ watch -d uptime </span><br><span class="line"> 10:30:57 up 98 days, 19:39,  3 users,  load average: 1.71, 0.75, 0.69</span><br></pre></td></tr></table></figure>

<p>3、开启第三个终端运行mpstat查看CPU使用率</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ mpstat -P ALL 5 20</span><br><span class="line">10:32:09 AM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle</span><br><span class="line">10:32:14 AM  all    6.80    0.00   33.75   26.16    0.00    0.39    0.00    0.00    0.00   32.90</span><br><span class="line">10:32:14 AM    0    4.03    0.00   69.57   19.91    0.00    0.00    0.00    0.00    0.00    6.49</span><br><span class="line">10:32:14 AM    1   25.32    0.00    9.49    0.00    0.00    0.95    0.00    0.00    0.00   64.24</span><br><span class="line">10:32:14 AM    2    0.24    0.00   10.87   63.04    0.00    0.48    0.00    0.00    0.00   25.36</span><br><span class="line">10:32:14 AM    3    1.42    0.00   36.93   14.20    0.00    0.28    0.00    0.00    0.00   47.16</span><br></pre></td></tr></table></figure>

<p>从这里可以看到，1分钟平均负载会慢慢增加到1.71，其中一个CPU的系统CPU使用率升到63.04。这说明，平均负载的升高是由于iowait升高。</p>
<p>那么我们到底是哪个进程导致的呢？我们使用pidstat来查看：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">$ pidstat -u 5 1</span><br><span class="line">Average:      UID       PID    %usr %system  %guest   %wait    %CPU   CPU  Command</span><br><span class="line">Average:        0         1    0.00    0.19    0.00    0.00    0.19     -  systemd</span><br><span class="line">Average:        0        10    0.00    0.19    0.00    1.56    0.19     -  rcu_sched</span><br><span class="line">Average:        0       599    0.58    1.75    0.00    0.39    2.33     -  systemd-journal</span><br><span class="line">Average:        0      1043    0.19    0.19    0.00    0.00    0.39     -  rsyslogd</span><br><span class="line">Average:        0      6934    0.00    1.56    0.00    1.17    1.56     -  kworker&#x2F;2:0-events_power_efficient</span><br><span class="line">Average:        0      7383    0.00    0.39    0.00    0.78    0.39     -  kworker&#x2F;1:0-events_power_efficient</span><br><span class="line">Average:        0      9411    0.00    0.19    0.00    0.58    0.19     -  kworker&#x2F;0:0-events</span><br><span class="line">Average:        0      9662    0.00   97.67    0.00    0.19   97.67     -  kworker&#x2F;u8:0+flush-253:0</span><br><span class="line">Average:        0     10793    0.00    0.97    0.00    1.56    0.97     -  kworker&#x2F;3:2-mm_percpu_wq</span><br><span class="line">Average:        0     11062    0.00   21.79    0.00    0.19   21.79     -  stress-ng-hdd</span><br><span class="line">Average:        0     11063    0.00    1.95    0.00    1.36    1.95     -  stress-ng-io</span><br><span class="line">Average:        0     11064    0.00    2.72    0.00    0.39    2.72     -  stress-ng-io</span><br><span class="line">Average:        0     11065    0.00    1.36    0.00    1.75    1.36     -  stress-ng-io</span><br><span class="line">Average:        0     11066    0.00    2.72    0.00    0.58    2.72     -  stress-ng-io</span><br></pre></td></tr></table></figure>

<p>可以发现是stress-ng导致的</p>
<h3 id="场景三、大量进程的场景"><a href="#场景三、大量进程的场景" class="headerlink" title="场景三、大量进程的场景"></a>场景三、大量进程的场景</h3><p>当系统中运行进程超出CPU运行能力时，就会出现等待CPU的进程。</p>
<p>比如：我们使用stress,但这次模拟8个进程：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ stress -c 8 --timeout 600</span><br></pre></td></tr></table></figure>

<p>我们的系统只有4颗CPU，这时候要运行8个进程，是明显不够的，系统的CPU后严重过载,这时候负载值达到了4点多：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$  uptime</span><br><span class="line"> 10:56:22 up 98 days, 20:05,  3 users,  load average: 4.52, 2.82, 2.67</span><br></pre></td></tr></table></figure>

<p>接着我们运行pidstat来查看一下进程的情况：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">$ pidstat -u 5 1</span><br><span class="line">Linux 5.0.5-1.el7.elrepo.x86_64 (k8s-m1)     07&#x2F;11&#x2F;2019     _x86_64_    (4 CPU)</span><br><span class="line"></span><br><span class="line">10:57:33 AM   UID       PID    %usr %system  %guest   %wait    %CPU   CPU  Command</span><br><span class="line">10:57:38 AM     0         1    0.20    0.00    0.00    0.00    0.20     1  systemd</span><br><span class="line">10:57:38 AM     0       599    0.00    0.99    0.00    0.20    0.99     2  systemd-journal</span><br><span class="line">10:57:38 AM     0      1043    0.60    0.20    0.00    0.00    0.79     1  rsyslogd</span><br><span class="line">10:57:38 AM     0     12927   51.59    0.00    0.00   48.21   51.59     0  stress</span><br><span class="line">10:57:38 AM     0     12928   44.64    0.00    0.00   54.96   44.64     0  stress</span><br><span class="line">10:57:38 AM     0     12929   45.44    0.00    0.00   54.56   45.44     2  stress</span><br><span class="line">10:57:38 AM     0     12930   45.44    0.00    0.00   54.37   45.44     2  stress</span><br><span class="line">10:57:38 AM     0     12931   51.59    0.00    0.00   48.21   51.59     3  stress</span><br><span class="line">10:57:38 AM     0     12932   48.41    0.00    0.00   51.19   48.41     1  stress</span><br><span class="line">10:57:38 AM     0     12933   45.24    0.00    0.00   54.37   45.24     3  stress</span><br><span class="line">10:57:38 AM     0     12934   48.81    0.00    0.00   50.99   48.81     1  stress</span><br><span class="line">10:57:38 AM     0     13083    0.00    0.40    0.00    0.20    0.40     0  pidstat</span><br></pre></td></tr></table></figure>

<p>可以看出，8个进程抢占4颗CPU，每个进程等到CPU时间(%wait)高达50%，这些都超出CPU计算能力的进程，最终导致CPU过载。</p>
 
      <!-- reward -->
      
      <div id="reword-out">
        <div id="reward-btn">
          打赏
        </div>
      </div>
      
    </div>
    

    <!-- copyright -->
    
    <div class="declare">
      <ul class="post-copyright">
        <li>
          <i class="ri-copyright-line"></i>
          <strong>版权声明： </strong>
          
          本博客所有文章除特别声明外，著作权归作者所有。转载请注明出处！
          
        </li>
      </ul>
    </div>
    
    <footer class="article-footer">
       
<div class="share-btn">
      <span class="share-sns share-outer">
        <i class="ri-share-forward-line"></i>
        分享
      </span>
      <div class="share-wrap">
        <i class="arrow"></i>
        <div class="share-icons">
          
          <a class="weibo share-sns" href="javascript:;" data-type="weibo">
            <i class="ri-weibo-fill"></i>
          </a>
          <a class="weixin share-sns wxFab" href="javascript:;" data-type="weixin">
            <i class="ri-wechat-fill"></i>
          </a>
          <a class="qq share-sns" href="javascript:;" data-type="qq">
            <i class="ri-qq-fill"></i>
          </a>
          <a class="douban share-sns" href="javascript:;" data-type="douban">
            <i class="ri-douban-line"></i>
          </a>
          <!-- <a class="qzone share-sns" href="javascript:;" data-type="qzone">
            <i class="icon icon-qzone"></i>
          </a> -->
          
          <a class="facebook share-sns" href="javascript:;" data-type="facebook">
            <i class="ri-facebook-circle-fill"></i>
          </a>
          <a class="twitter share-sns" href="javascript:;" data-type="twitter">
            <i class="ri-twitter-fill"></i>
          </a>
          <a class="google share-sns" href="javascript:;" data-type="google">
            <i class="ri-google-fill"></i>
          </a>
        </div>
      </div>
</div>

<div class="wx-share-modal">
    <a class="modal-close" href="javascript:;"><i class="ri-close-circle-line"></i></a>
    <p>扫一扫，分享到微信</p>
    <div class="wx-qrcode">
      <img src="//api.qrserver.com/v1/create-qr-code/?size=150x150&data=http://example.com/2020/11/11/linux/%E6%80%8E%E4%B9%88%E7%90%86%E8%A7%A3linux%E7%9A%84%E5%B9%B3%E5%9D%87%E8%B4%9F%E8%BD%BD%E5%8F%8A%E5%B9%B3%E5%9D%87%E8%B4%9F%E8%BD%BD%E9%AB%98%E5%90%8E%E7%9A%84%E6%8E%92%E6%9F%A5%E5%B7%A5%E5%85%B7/" alt="微信分享二维码">
    </div>
</div>

<div id="share-mask"></div>  
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/linux/" rel="tag">linux</a></li></ul>

    </footer>
  </div>

   
  <nav class="article-nav">
    
      <a href="/2020/11/11/linux/%E5%88%A9%E7%94%A8%20Shell%20%E8%84%9A%E6%9C%AC%E5%AE%9E%E7%8E%B0%E9%82%AE%E4%BB%B6%E7%9B%91%E6%8E%A7%20Linux%20%E7%B3%BB%E7%BB%9F%E7%9A%84%E5%86%85%E5%AD%98/" class="article-nav-link">
        <strong class="article-nav-caption">上一篇</strong>
        <div class="article-nav-title">
          
            利用 Shell 脚本实现邮件监控 Linux 系统的内存.md
          
        </div>
      </a>
    
    
      <a href="/2020/11/11/mysql/01.MySQL%E4%B9%8Bmysqldump%E7%9A%84%E4%BD%BF%E7%94%A8/" class="article-nav-link">
        <strong class="article-nav-caption">下一篇</strong>
        <div class="article-nav-title">01.MySQL之mysqldump的使用.md</div>
      </a>
    
  </nav>

   
<!-- valine评论 -->
<div id="vcomments-box">
  <div id="vcomments"></div>
</div>
<script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/valine@1.4.14/dist/Valine.min.js"></script>
<script>
  new Valine({
    el: "#vcomments",
    app_id: "",
    app_key: "",
    path: window.location.pathname,
    avatar: "monsterid",
    placeholder: "给我的文章加点评论吧~",
    recordIP: true,
  });
  const infoEle = document.querySelector("#vcomments .info");
  if (infoEle && infoEle.childNodes && infoEle.childNodes.length > 0) {
    infoEle.childNodes.forEach(function (item) {
      item.parentNode.removeChild(item);
    });
  }
</script>
<style>
  #vcomments-box {
    padding: 5px 30px;
  }

  @media screen and (max-width: 800px) {
    #vcomments-box {
      padding: 5px 0px;
    }
  }

  #vcomments-box #vcomments {
    background-color: #fff;
  }

  .v .vlist .vcard .vh {
    padding-right: 20px;
  }

  .v .vlist .vcard {
    padding-left: 10px;
  }
</style>

 
     
</article>

</section>
      <footer class="footer">
  <div class="outer">
    <ul>
      <li>
        Copyrights &copy;
        2015-2020
        <i class="ri-heart-fill heart_icon"></i> TzWind
      </li>
    </ul>
    <ul>
      <li>
        
        
        
        由 <a href="https://hexo.io" target="_blank">Hexo</a> 强力驱动
        <span class="division">|</span>
        主题 - <a href="https://github.com/Shen-Yu/hexo-theme-ayer" target="_blank">Ayer</a>
        
      </li>
    </ul>
    <ul>
      <li>
        
        
        <span>
  <span><i class="ri-user-3-fill"></i>访问人数:<span id="busuanzi_value_site_uv"></span></s>
  <span class="division">|</span>
  <span><i class="ri-eye-fill"></i>浏览次数:<span id="busuanzi_value_page_pv"></span></span>
</span>
        
      </li>
    </ul>
    <ul>
      
    </ul>
    <ul>
      
    </ul>
    <ul>
      <li>
        <!-- cnzz统计 -->
        
        <script type="text/javascript" src='https://s9.cnzz.com/z_stat.php?id=1278069914&amp;web_id=1278069914'></script>
        
      </li>
    </ul>
  </div>
</footer>
      <div class="float_btns">
        <div class="totop" id="totop">
  <i class="ri-arrow-up-line"></i>
</div>

<div class="todark" id="todark">
  <i class="ri-moon-line"></i>
</div>

      </div>
    </main>
    <aside class="sidebar on">
      <button class="navbar-toggle"></button>
<nav class="navbar">
  
  <div class="logo">
    <a href="/"><img src="/images/ayer-side.svg" alt="Hexo"></a>
  </div>
  
  <ul class="nav nav-main">
    
    <li class="nav-item">
      <a class="nav-item-link" href="/">主页</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/archives">归档</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/categories">分类</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/tags">标签</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" target="_blank" rel="noopener" href="http://www.baidu.com">百度</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/friends">友链</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/2019/about">关于我</a>
    </li>
    
  </ul>
</nav>
<nav class="navbar navbar-bottom">
  <ul class="nav">
    <li class="nav-item">
      
      <a class="nav-item-link nav-item-search"  title="搜索">
        <i class="ri-search-line"></i>
      </a>
      
      
      <a class="nav-item-link" target="_blank" href="/atom.xml" title="RSS Feed">
        <i class="ri-rss-line"></i>
      </a>
      
    </li>
  </ul>
</nav>
<div class="search-form-wrap">
  <div class="local-search local-search-plugin">
  <input type="search" id="local-search-input" class="local-search-input" placeholder="Search...">
  <div id="local-search-result" class="local-search-result"></div>
</div>
</div>
    </aside>
    <script>
      if (window.matchMedia("(max-width: 768px)").matches) {
        document.querySelector('.content').classList.remove('on');
        document.querySelector('.sidebar').classList.remove('on');
      }
    </script>
    <div id="mask"></div>

<!-- #reward -->
<div id="reward">
  <span class="close"><i class="ri-close-line"></i></span>
  <p class="reward-p"><i class="ri-cup-line"></i>请我喝杯咖啡吧~</p>
  <div class="reward-box">
    
    
  </div>
</div>
    
<script src="/js/jquery-2.0.3.min.js"></script>


<script src="/js/lazyload.min.js"></script>

<!-- Tocbot -->


<script src="/js/tocbot.min.js"></script>

<script>
  tocbot.init({
    tocSelector: '.tocbot',
    contentSelector: '.article-entry',
    headingSelector: 'h1, h2, h3, h4, h5, h6',
    hasInnerContainers: true,
    scrollSmooth: true,
    scrollContainer: 'main',
    positionFixedSelector: '.tocbot',
    positionFixedClass: 'is-position-fixed',
    fixedSidebarOffset: 'auto'
  });
</script>

<script src="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.js"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.css">
<script src="https://cdn.jsdelivr.net/npm/justifiedGallery@3.7.0/dist/js/jquery.justifiedGallery.min.js"></script>

<script src="/dist/main.js"></script>

<!-- ImageViewer -->

<!-- Root element of PhotoSwipe. Must have class pswp. -->
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

    <!-- Background of PhotoSwipe. 
         It's a separate element as animating opacity is faster than rgba(). -->
    <div class="pswp__bg"></div>

    <!-- Slides wrapper with overflow:hidden. -->
    <div class="pswp__scroll-wrap">

        <!-- Container that holds slides. 
            PhotoSwipe keeps only 3 of them in the DOM to save memory.
            Don't modify these 3 pswp__item elements, data is added later on. -->
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>

        <!-- Default (PhotoSwipeUI_Default) interface on top of sliding area. Can be changed. -->
        <div class="pswp__ui pswp__ui--hidden">

            <div class="pswp__top-bar">

                <!--  Controls are self-explanatory. Order can be changed. -->

                <div class="pswp__counter"></div>

                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>

                <button class="pswp__button pswp__button--share" style="display:none" title="Share"></button>

                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>

                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>

                <!-- Preloader demo http://codepen.io/dimsemenov/pen/yyBWoR -->
                <!-- element will get class pswp__preloader--active when preloader is running -->
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                        <div class="pswp__preloader__cut">
                            <div class="pswp__preloader__donut"></div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div>
            </div>

            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>

            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>

            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>

        </div>

    </div>

</div>

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/default-skin/default-skin.min.css">
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe-ui-default.min.js"></script>

<script>
    function viewer_init() {
        let pswpElement = document.querySelectorAll('.pswp')[0];
        let $imgArr = document.querySelectorAll(('.article-entry img:not(.reward-img)'))

        $imgArr.forEach(($em, i) => {
            $em.onclick = () => {
                // slider展开状态
                // todo: 这样不好，后面改成状态
                if (document.querySelector('.left-col.show')) return
                let items = []
                $imgArr.forEach(($em2, i2) => {
                    let img = $em2.getAttribute('data-idx', i2)
                    let src = $em2.getAttribute('data-target') || $em2.getAttribute('src')
                    let title = $em2.getAttribute('alt')
                    // 获得原图尺寸
                    const image = new Image()
                    image.src = src
                    items.push({
                        src: src,
                        w: image.width || $em2.width,
                        h: image.height || $em2.height,
                        title: title
                    })
                })
                var gallery = new PhotoSwipe(pswpElement, PhotoSwipeUI_Default, items, {
                    index: parseInt(i)
                });
                gallery.init()
            }
        })
    }
    viewer_init()
</script>

<!-- MathJax -->

<!-- Katex -->

<!-- busuanzi  -->


<script src="/js/busuanzi-2.3.pure.min.js"></script>


<!-- ClickLove -->

<!-- ClickBoom1 -->

<!-- ClickBoom2 -->

<!-- CodeCopy -->


<link rel="stylesheet" href="/css/clipboard.css">

<script src="https://cdn.jsdelivr.net/npm/clipboard@2/dist/clipboard.min.js"></script>
<script>
  function wait(callback, seconds) {
    var timelag = null;
    timelag = window.setTimeout(callback, seconds);
  }
  !function (e, t, a) {
    var initCopyCode = function(){
      var copyHtml = '';
      copyHtml += '<button class="btn-copy" data-clipboard-snippet="">';
      copyHtml += '<i class="ri-file-copy-2-line"></i><span>COPY</span>';
      copyHtml += '</button>';
      $(".highlight .code pre").before(copyHtml);
      $(".article pre code").before(copyHtml);
      var clipboard = new ClipboardJS('.btn-copy', {
        target: function(trigger) {
          return trigger.nextElementSibling;
        }
      });
      clipboard.on('success', function(e) {
        let $btn = $(e.trigger);
        $btn.addClass('copied');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-checkbox-circle-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPIED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-checkbox-circle-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
      clipboard.on('error', function(e) {
        e.clearSelection();
        let $btn = $(e.trigger);
        $btn.addClass('copy-failed');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-time-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPY FAILED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-time-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
    }
    initCopyCode();
  }(window, document);
</script>


<!-- CanvasBackground -->


    
  </div>
</body>

</html>